Securing Commercial Operating Systems

Since the discovery of the reference monitor concept during the development of Multics, there have been many projects to retrofit existing commercial operating systems with a true reference monitor implementation. Successful, commercial operating systems can have a large customer base and a variety of popular applications. As a result, those customers with strong secrecy and integrity requirements (e.g., US Government) often encourage the construction of secure versions of existing commercial operating systems. Many such systems have been retrofitted over the years. In this chapter, we explore some of the commercial systems that have been retrofitted with reference monitors. The aim is not for completeness, as there are far too many systems, but we want to capture the distinct movements in creating a secure operating system from an existing commercial system. Converting an existing code base to one that implements a reference monitor is a challenging task. In order to be a secure operating system, the resulting code base must achieve the three reference monitor guarantees, but this is difficult because much of the code was not developed with these guarantees in mind. This contrasts markedly with the security kernel approach in Chapter 6 where the system design considers mediation, tamperproofing, and verification from the outset. After outlining the tasks involved in retrofitting a commercial operating system with a reference monitor, we examine a variety of different retrofitted systems. We group these systems by a trend motivating their construction. We examine the resultant system architectures in detail for two systems: Solaris Trusted Extensions in Chapter 8 and the Linux operating system in Chapter 9.